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Response to Arguments 
Applicant's arguments filed 9/28/2004 have been fully considered but they are not 
persuasive. 

Applicant on pages 11-12 third paragraph argues the term "search performance" is not 
essential to the practice of the invention, the search performance is just one of the advantage of 
certain embodiments of the invention. Therefore, the Examiner should be withdrawing the 
rejection 35 USC §112 first paragraph. Examiner's reply: The examiner withdraws the rejection 
under 35 USC §112 first paragraph, in respect to the Applicant's feedback on the "search 
performance". 

Applicant on pages 12-13 argues in response to Examiner's question: What does 
Applicant try to claim in the claims 30 and 35 when N=l? Examiner acknowledges Applicant's 
respond of only one color is selected for extraction from the region when N=l. Examiner's reply: 
the rejection of claims 30 and 35 under 35 USC SI 12 second paragraph has been withdrawn. 
Applicant on page 15, second paragraph argues the reference Graham fails to disclose dominant 
color setting method, which targets content-based retrieval for color in visual data. Examiner's 
reply: Note: Visual element descriptor is considered to be color, spatial structure, shape and 
motion. Color composes of color space that is RGB, YCrCb, HSV (hue, saturation, value) and 
linear transformation matrix. Dominant color is to specify set of dominant color in a shape 
region, use color histogram and content-based retrieval. Color histogram is the percent of each 
color. Graham in abstract teaches that a system and method for finding areas of similar color 
(ROI) in electronically captured spot color images and replacing the similar colors with a single 
dominant color. Graham in col. 3, lines 66-67 discloses a pallet of dominant colors is identified 
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based on the histogram classification. The color pallet is used to identify the dominant colors in 
the image. Graham in col, 4, lines 11-15 discloses the spatial areas are finally compared with 
the pallet of colors and the average noted color of the spatial areas are replaced with the closest 
corresponding pallet color. Graham in the abstract teaches each line of the original image is 
then scanned at a second resolution, the second resolution being substantially higher than the 
first resolution. The scanned lines are then processed to determine where within each line 
different colors exist. The locations of the different colors are noted using a break out box 
comprising a set of rules to evaluate each pixel . Graham illustrates in figs. 14 and 15 the 
horizontal and vertical skew correction in conjunction with the invention. But does not 
explicitly specify an accuracy of a color value representing the region. 
Applicant on page 16 lines 3-11 argues that Graham is directed to a spot color extraction 
method and not to image characterization for image indexing and retrieval. Examiner's 
interpretation: each image is made of a plurality of spot color (pixel), Graham in the abstract 
discloses a system and method for finding areas of similar color in electronically captured spot 
color images (plural) and replacing the similar colors (plural) with a single dominant color. 
Applicant on page 16, lines 9-1 1 argues the Examiner's rationale for modifying Graham in view 
of Qian is not applicable. Examiner's reply: Applicant on page 25 of specification lines 1-5 
discloses the size of a region is computed by the counting of the pixels in the region. Applicant 
on page 11, lines 13-18 discloses a filter method for smoothing and burring a region when 
obtaining a dominant value. From the last paragraph, a person skill in the art would be 
associating the conserving computation/storage/noise to modify Graham in view of Qian, that is 
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to be invariant to certain types of transformation of the image, conserve computational and 
storage resources, be insensitive to noise. 

Applicant on page 16 lines 16-21 argues the Graham fails to disclose a confidence measure 
extraction method. Examiner's reply: Graham in fig. 8 A illustrates the box 816 represents an 
intensity tolerance T, and D represents the distance from the center to the back or front of the 
box measured in pixels. 

Applicant on page 16 regarding claim 22 argues the Graham does not disclose determining a 
count sum of a confidence and pixels as an initial value corresponding to each region dominant 
color with respect to all region dominant colors and multiplying the coherence value and color 
pixel, adding a confidence to the multiplied value .... Examiner's reply: the rejecfion of claim 
22 has been withdrawn. 

Applicant on page 17 second paragraph regarding claim 25 argues the Graham does not disclose 
... method between different retrieval systems, .. Examiner's reply: Graham in fig. 1 illustrates 
different system, scanner, printer, and a laptop. Graham in col. 5 lines 10-21 teaches the system 
processes 2 RGB data formats. One is 24-bit (different format) where there is a red, a green, 
and a blue byte. The other one is 32-bit (different format) where there is an alpha byte, a red, a 
green, and a blue. The alpha byte is actually unused. The 32-bit scheme is convention on the 
Macintosh.RTM., and permits loading and storage of RGB pixels as one 32-bit integer to speed- 
up processing. These type of data are processes and output as pallet images. A pallet image 
comprises an 8-bit (different format) per pixel image, thus, each byte represents one pixel. As 
discussed above, each 8-bit word is an index into a pallet of colors. Graham in fig. 2 box 210 
illustrates comparing data. Graham in cols. 1 and 2 lines 66-67; 1-7 respectively discloses the 
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process of posterization is basically one of taking that true color image and converting 
(transforming) it to the closest match in a pallet by substituting the RGB pixel with the pixel in 
the pallet, or in the case of converting it to a pallet image, to replace it in the index, Graham in 
col. 8 lines 23-32 teaches conceptually, if the histogram has a peak, then a valley and another 
peak, the peak has a maximum number of values. As the topography is traversed, the cells will 
have lesser and lesser values in them until the bottom of the valley is reached. Then the search 
starts to go up the next slope, and the number of occurrences will go up. The search in that 
direction then stops, and all cells down until the valley are included. Going up again indicates 
that the search reaches another color cloud. Graham in fig. 1 box 102 illustrates sharing data 
format between a scanner and a laptop. 

Applicant on page 17 third paragraph regarding claim 27 argues that Graham fails to disclose a 
method for describing dominant color of visual data. Examiner's reply: Graham in col. 3 lines 
65-67 teaches a pallet of dominant colors is identified based on the histogram classification. 
The color pallet is used to identify the dominant colors in the image. 
Applicant on page 17 the same paragraph argues the Graham does not disclose selecting a 
region of interest from media object. Examiner's reply: Graham in col. 6 lines 25-34 teaches 
using well-known techniques; the user is permitted to select an area, which he may want to 
perform the spot color extraction on. Alternatively, the entire prescaimed spot image may be 
selected. 

Applicant on pages 17 and 18 argues the reference Graham does not disclose the limitations in 
claim 27. Examiner's reply: Graham in fig, 9 illustrates number of colors (N), and there is more 
one color that means N>1. For the rest of the limitations see claim 1 rejection. 



Application/Control Number: 09/609,392 Page 6 

Art Unit: 2672 

Applicant on page 18 lines 6-8 regarding claim 27 argues the reference Qian fails to overcome 
the deficiencies of Graham. Examiner's reply: The reference Graham is silent about accuracy of 
a representative color value, however Qian in fig. 10, to overcome these limitations and to 
maintain a relatively compact matrix, the present inventors determined that the matrix should 
include smaller percentage ranges at the smaller percentages, with increasing percentage ranges 
toward the larger percentages. This maintains a small matrix, which is suitable for embedded 
systems, while providing more accurate discrimination between images with similar color 
content. 

Applicant on pages 18-19 argues substantially similar to the previous arguments in this office 
action. 

Examiner's suggestion: Applicant should be selecting detailed claim language, the present 
claim language is very broad and dominating the references claim invention. Examiner 
encourages Applicant to schedule an interview. 

Allowable Subject Matter 

Claim 22 allowed. 

The following is an examiner's statement of reasons for allowance: multiplying the 
coherence value and the color pixeK adding a confidence to the multiplied value and 
obtaining a confidence with respect to the region dominant color; and dividing the thuslv 
obtained confidence value by a region size and extracting a confidence with respect to the 
image region. 
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Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claim 1-12, 25 and 27-37 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Graham et al. (herein after refers as a Graham), and further in view of Qian et al. (hereinafter 
refers as a Qian). 
1. Claim 1, 

"A dominant color setting method which generating a region dominant color descriptor 
incorporating information indicating a number of dominant colors with respect to a region of 
interest in visual data, at least one expressed dominant color, a firequency with which the 
dominant color appears in the region, and an accuracy of a color value representing the region". 
[Examiner's comments: Visual element descriptor is considered to be color, spatial structure, 
shape and motion. Color composes of color space that is RGB, YCrCb, HS V (hue, saturation, 
value) and linear transformation matrix. Dominant color is to specify set of dominant color in a 
shape region, use color histogram and content-based retrieval. Color histogram is the percent of 
each color.] Graham in abstract teaches that a system and method for finding areas of similar 
color (ROI) in electronically captured spot color images and replacing the similar colors with a 
single dominant color. Graham in col. 3, lines 66-67 discloses a pallet of dominant colors is 
identified based on the histogram classification. The color pallet is used to identify the 
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dominant colors in the image. Graham in coL 4, lines 11-15 discloses the spatial areas are 
finally compared with the pallet of colors and the average noted color of the spatial areas are 
replaced with the closest corresponding pallet color. Graham does not explicitly specify 
accuracy information. However Qian in col. 2, lines 7-21 teaches the coherence of the color of a 
picture element in relation to that of other picture elements in a contiguous region is 
determined. Even though the number of picture elements of each color is equal and, therefore, 
the color histograms are identical for two images, differences between features in the images 
will mean that the numbers of picture elements of each color, which are color coherent, will 
vary. Color coherence vectors do embed some spatial information in the descriptors. Qian in 
col. 4, lines 63-67 teaches the mean values of the individual red, green, and blue (RGB) pixels, 
a transform of the RGB pixel values or the mean color or the vector sum of the RGB intensity 
values might be used to describe the color of a test area of heterogeneous color. Also Qian in 
fig. 10, to overcome these limitations and to maintain a relatively compact matrix, the present 
inventors determined that the matrix should include smaller percentage ranges at the smaller 
percentages, with increasing percentage ranges toward the larger percentages. This maintains a 
small matrix, which is suitable for embedded systems, while providing more accurate 
discrimination between images with similar color content. Thus, it would have been obvious to 
one of ordinary skill in the art at the time the invention was made to incorporate the teaching of 
Qian into Graham in order to be invariant to certain types of transformations of the image, 
conserve computational and storage resources, be insensitive to noise, providing more accurate 
discrimination between similar color content and be easy to interpret in a normal sense. 
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2. Regarding Claims 2 and 10, Graham in col. 3, line 1 indicates the number of different 
color may actually be 8000 to choose from. And also see rejection of claim 1. The step of this 
claim the first dominant color descriptor and the second dominant color descriptor is obvious 
because applicant does not specify set of DC in a shape region, and the first dominant color 
descriptor could be a background color, and the second DC could be a shape region. 

3. Claims 3 and 11 

An expression of accuracy of the dominant colors as extracted by a certain method is obtained 
in accordance with a degree of confidence of the region dominant color descriptor. Graham in 
col. 5 lines 57-64 discloses the dominant colors are accomplished by creating "clusters" of 
colors that are related. This is done by taking the maximum occurrence and finding the colors 
around that maximum occurrence that are related and including the related colors in the color 
cluster. See rejection of claim 1. 

4. Claim 4, 

Graham in figs. 10 and 1 1 illustrates the broad claim language in claim 4. And also Graham in 
col. 4 lines 9-15 discloses Segments of each line are correlated with each other and with 
segments of adjacent lines to determine if the noted colors are in similar locations to thereby 
identify various spatial areas of color representing the original. The spatial areas are finally 
compared with the pallet of colors and the average noted color of the spatial areas is replaced 
with the closest corresponding pallet color. 

5. Claim 5, 

Graham in fig. 10 steps 1004-1016 illustrates matching the current color to the pallet color and 
resolved areas vertically and horizontally. 
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6. Claim 6. 

A coherency value is used to represent the concentration degree of the pixels of a color with 
respect to the dominant color is adapted to the confidence. Graham in col 4, lines 5-7 discloses 
the locations of the different colors are noted (e.g., stored in memory), using a break out box 
comprising a set of rules to evaluate each pixel. 

7. Claims 7 and 8, 

The step of claims 7 and 8 are obvious because Graham in figs. 14-15 illustrates an example of a 
horizontal and vertical skew correction. 

8. Claim 9,Graham does not explicitly specify accuracy information includes to a 
confidence measure expressed by a vector value based on coherence value. However Qian in col. 
2, lines 7-21 teaches the coherence of the color of a picture element in relation to that of other 
picture elements in a contiguous region is determined. Even though the number of picture 
elements of each color is equal and, therefore, the color histograms are identical for two images, 
differences between features in the images will mean that the numbers of picture elements of 
each color, which, are color coherent, will vary. Color coherence vectors do embed some spatial 
information in the descriptors. Qian in col. 4, lines 63-67 teaches the mean values of the 
individual red, green, and blue (RGB) pixels, a transform of the RGB pixel values or the mean 
color or the vector sum of the RGB intensity values might be used to describe the color of a test 
area of heterogeneous color. Thus, it would have been obvious to one of ordinary skill in the art 
at the time the invention was made to incorporate the teaching of Qian into Graham in order to to 
be invariant to certain types of transformations of the image, conserve computational and storage 
resources, be insensitive to noise, and be easy to interpret in a normal sense. 
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9. Claims 10, 11 and 12 

Applicant does not specify explicitly the claim invention. Applicant repeats the same 
procedures in claims. See rejection of claim 1. 

10. The Hmitation as recited in claim 25 that is the method of transforming and searching a 
sharing data format using a region descriptor of each system. However this additional featured 
imitation of transforming and searching is described in the abstract of Graham Segments of each 
line are correlated with each other and with segments of adjacent lines to determine if the noted 
colors are in similar locations to thereby identify various spatial areas of color representing the 
original The spatial areas are finally compared with the pallet of colors and the average noted 
colors of the spatial areas are replaced with the closest corresponding pallet color. 

11. Claim 27, A method for describing dominant color of visual data, comprising: selecting a 
region of interest fi-om a media object; and generating a dominant color descriptor for the region, 
said descriptor including": a) information indicative of a number of colors (N) selected for 
extraction fi-om the region, where N _> 1"; b) information indicative of color values determined 
for respective ones of the N colors; c) information indicative of frequencies with which 
respective ones of the N colors appear in the region; d) information indicative of an accuracy of 
a representative color value for the region, said representative color value determined based on 
the information in at least one of b) and c).See rejection of claim 1. 

12. Claim 28, The method of claim 27, wherein the color values in b) are determined based 
on any one of the following extraction methods: an average-color method; a method of 
expressing only one most frequency appearing color in the region; a method of expressing more 
than one most frequently appearing color in the region; a method of determining which colors 
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appear in the region more than a predetermined percentage of a threshold value; and a histogram 
method". Graham in col. 3, lines 53-68 discloses that the dimensions of the histogram may 
comprise other known image coordinate systems. A pallet of dominant colors is identified based 
on the histogram classification. The color pallet is used to identify the dominant colors in the 
image. 

13. Claim 29, The method of claim 27, wherein the information in c) is determined 
based on pixel counts for respective ones of the N colors". Graham in abstract teaches this 
limitation. 

14. Claim 30, see rejection of claim 1 . 

15. Claim 3 1 , The method of claim 27, wherein the color values in b) are defined by at 
least one of color space information, Quantization information, color clustering 
information, and channel information. Graham in col. 5, lines 57-64 teaches the limitation. 

16. Claim 32, see rejection of claim 12. 

17. Claim 33, A method for describing dominant color in visual data, comprising: selecting 
a region of interest fi:'om a media object; and generating a dominant color descriptor for the 
region, said descriptor including": a) information indicative of at least one color selected for the 
region, b) information indicative of accuracy of a color value assigned to the region, said color 
value based on the information in (a). See rejection of claim 27. 

18. Claim 34, It is obvious because, "A computer-readable medium for setting color 
information for visual data, having stored thereon": a) information indicative of a number of 
colors (N) selected for extraction firom a region of a media object, where N >_ 1; b) information 
indicative of color values determined for respective ones of the N colors; c) information 
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indicative of frequencies with which respective ones of the N colors appear in the region"; d) 
information indicative of an accuracy of a representative color value for the region, said 
representative color value determined based on the information in at least one of b) and c). See 
rejection of claim 27. 

19. Claim 35, see rejection of claim 1. 

20. Claim 36, "The medium of claim 34, wherein the color values in b) are defined by at 
least one of color space information, quantization information, color clustering information, and 
channel information". Graham in col. 5, lines 57-64 teaches the limitation. 

21 . Claim 37, see rejection of claim 32. 

Conclusion 

THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 
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Any inquiry concerning this communication or earlier conmiunications from the 
examiner should be directed to Javid A Amini whose telephone number is 703-605-4248. The 
examiner can normally be reached on 8-4pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Michael Razavi can be reached on 703-305-4713. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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